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Introducing Ring -3 



Getting there 



Writing useful Ring -3 rootkits 




A Quest to Ring -3 




Usermode rootkits 



Kernelmode rootkits 



ypervisor rootkits (Bluepill) 



SMM rootkits 
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Many (all?) vPro chipsets (MCHs) have: 

• An Independent CPU (not IA32!) 

v' Access to dedicated DRAM memory 

v' Special interface to the Network Card (NIC) 

s Execution environment called Management Engine (ME) 



Your chipset is a little computer. It can execute programs in 
parallel and independently from the main CPU! 



On the SPI-flash chip (the same one used for the BIOS code) 



It is a separate chip on a motherboard: 
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Of course one cannot reflash the SPI chip at will! 

vPro-compatible systems do not allow unsigned updates to its firmware (e.g. BIOS reflash). 



But see our talk tomorrow about breaking into the Intel BIOS ;) 
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Anyway: 
The chipset runs programs. 

The programs are stored in the (well protected) flash 
memory, together with BIOS firmware. 



Intel Active Management Technology (AMT) 



http: //www. intel.com/technology/platform-technology/intel-amt/ 
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lntel ,]|> Active Management Technology 

Computer: iDBO 
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System 

Processor 

Memory 

Disk 
Event Log 
Remote Control 
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User Accounts 



Processor Information 

Processor 1 
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Socket 

Version 
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Upgrade method 

Populated? 




Done 
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Interactive Management Technology 

Computer: iDBO 



System Status 
Hardware Information 

System 

Processor 

Memory 

Disk 
Event Log 
Remote Control 
Power Policies 
Network Settings 
User Accounts 



Remote Control 




Done 



(^Manageability Commander Tool 



File Edit View Help 




O Intel® AMT Certificate Store - 192.168.0.22 
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Certificates | Trusted Roots | Revocation 
Stored Certificates - 



Certificate 



New. 



Import. 



Remove 



View.. 



Close 
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Connect & Control 

In this window, you can connect to an Intel® AMT computer. 
Once connected, you can control the computer remotely, 
remotely turn on or off the Intel® AMT computer you are 
connected to, control network policies and filters, boot the 
computer to a remote drive, view the hardware asset inventory, 
and read the computer's event log. 



Connection | Remote Control Intel® Management Engine | Networking | 






inter 6 ' Management tngine 
Computer Hostname / Domain 


iDBO. somedomain. org * 1 


Intel® AMT Version 


3.2.1 


BIOS Version 


J0Q351 QJ.86A.0S33.200S.0707.224S 


Intel® Management Engine Pi 


ON in SO _J 


User Accounts 


2 User Accounts * I 


Interaction Type 


EOI (SOAP) only _J 


Certificate & CRL Store 


certificate(s), trusted root(s) I * I 


Kerberos* Setup 


Disabled * I 


Remote Access 


Unsupported x I 



Set Time... 



WSMAN Browser. 



Manageability Commander Tool - vO. 6. 0937. 2 




O Manageability Commander Tool 



Edit View Help 

Network 
&■■■] : 192.1 63.0.22/ admin 
' 192.168.0.66/ admin 



- □ X 



Connect & Control 

In this window, you can connect to an Intel® AMT computer. 
Once connected, you can control the computer remotely, 



(J> Manageability Terminal Tool - 192.168.0.22 



Terminal Edit 
Serial-over-LAN 



TCP Redirect 

No Mapping 
Ok/Ok 



Remote Command 
Normal Reboot 
Power Up 
Power Down 
Remote Reboot 
Remote Reboot to BIOS Setup 



Disk Redirect Serial Agent 



NTH 



Full power (50) 



TV, 




1 





Networking | 



iDBO. somedomain. org 

3.2.1 

JOQ3510J. S6A. 0933. 2003. 0707. 2243 

ON in SO 

2 User Accounts 

EOI (SOAP) only 

certificate(s), trusted root(s] 

Disabled 

Unsupported 



Floppy f4.img 

CDROM fc9_iso 

IDE Redirect Active: bytes Sent / bytes Received 



vO. 6. 0937. 2 



Manageability Commander Tool 



File Edit View Help 



-|D| x| 



El~iJ Network 

S- : 192.168.0.22 /admin 
' 192.168.0.66/ admin 



O Manageability Terminal Tool - 192.168.0.22 



Terminal Edit Remote Command Disk Redirect 
Serial-over-LAN - Connected 



Connect & Control 

In this window, you can connect to an Intel® AMT computer. 
Once connected, you can control the computer remotely, 
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Serial Agent 



>ry. 



Full power (SO) 




System Setup 



Advance* 



Peripheral Configuration 
Drive Configuration 
Event Log Configuration 
Video Configuration 
Fan Control 
Hardware Monitoring 
Chipset Configuration 
USB Configuration 



Power 



Intel (R) HE Exit 



Setup Warning: 

Setting items on this Screen to incorrect values 

may cause system to malfunction! 



<>=Select Screen 
I i=Select Item 
Enter=Select Submenu 
F9=Setup Defaults 
F10=Save and Exit 
Esc=Previous Page 
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Networking | 



iDBO. somedomain. org 

3.2.1 

JOQ351 OJ.36A. 0933. 2008. 0707. 2248 

ON in SO 

2 User Accounts 

EOI (SOAP) only 

certificate(s), trusted root(s] 

Disabled 

Unsupported 



TCP Redirect 

No Mapping 
Ok/Ok 

vO. 6. 0937. 2 




Floppy f4.img 

CDROM fc9iso 
IDE Redirect Disabled 
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■anageability Commander Tool 



Help 



- □ X 



E\->_J Network 

m- ' 192.1 63.0.22 /admin 
192.1 68.0.66 /admin 



Connect & Control 

In this window, you can connect to an Intel^ AMT computer. 
Once connected, you can control the computer remotely. 



O Manageability Terminal Tool - 192.168.0.22 



Terminal Edit Remote Command Disk Redirect 
Serial-over-LAN - Connected 



ISOLINUX 



Serial Agent 



[1] RescueSysteni 
[2] RescueSysteni 
[3] memtestSS 



- load cd into RAH 



Loading initrdx 
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Full power (SO) 



Peter: Anvin 




Networking | 



iDBO. somedomain. org 
3.2.1 

JOQ351 0J.S6A.0333.2003.0707.2243 

ON in SO 

2 User Accounts 

EOI [SOAP] only 

certificate(s), trusted root(s) 

Disabled 

Unsupported 



TCP Redirect 

No Mapping 
Ok/Ok 



IDE Redirect 



<&»> 



Floppy f4.img 

CDROM fc9.iso 

IDE Redirect Active: 9663504 bytes 9ent / bytes Received 



vO. 6. 0937. 2 



If abused, AMT offers powerful backdoor capability: 
it can survive OS reinstall or other OS change! 



There are a few methods to enable AMT... 

... but most require physical presence during the BIOS boot 

We do have ideas how to do it remotely, 

But let's skip it and talk about something better... 



But turns out that some AMT code is executed regardless of 

whether AMT is enabled in BIOS or not! 
And we can hook this code (see later)! 







Injecting Code into AMT/ME 



Ok, so how we get our code executed inside AMT/ME environment? 



Memory Remapping on Q35 chipset 



TOUUD 



REMAPLIMIT 



REMAPBASE 



4GB 
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MMIO 



5GB 



This DRAM now accessible from 
CPU at physical addresses: 

<REMAPBASE, REMAPLIMIT> 
Otherwise it would be wasted! 



Processor's View 



DRAM 



remap_base 

remap_limit 

touud 



= 0x100000000 (4G) 
= 0xl83ffffff 
= 0x184000000 



reclaim_mapped_to = 0x7c000000 
AMT normally at: 0x7 f 000000, 

NOW remapped tO : 0x103000000 (and freely accessible by the OS!) 



(Offsets for a system with 2GB of DRAM) 



Fixed? No problem - just revert to the older BIOS! 

(turns out no user consent is needed to downgrade Intel BIOS to an earlier version - malware can 

perfectly use this technique, it only introduces one additional reboot) 



This attack doesn't work against the Intel Q45-based boards. 
The AMT region seems to be additionally protected. 

(We are investigating how to get access to it...) 
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Writing Useful Ring -3 Rootkits 



Independent of main CPU 

Can access host memory via DMA (with restrictions) 

Dedicated link to NIC, and its filtering capabilities 

Can force host OS to reboot at any time (and boot the 

system from the emulated CDROM) 

Active even in S3 sleep! 



A few words about the ARC4 processor (integrated in the MCH) 

RISC architecture 

^ 32-bit general purpose registers and memory space 

& "Auxiliary" registers space, which is used to access hardware 

On Q35 boards, the 0x01000000-0x02000000 memory 

range (of the ARC4 processor) is mapped to the top 16MB of 

host DRAM 



The ARC compiler suite (arc-gnu -too Is) used to be freely available (a 

few months ago)... 

Now it seems to be a commercial product only: 

http: //www. arc.com/software/qnutools/ 



(we were luckily enough to download it when it was still free) 



Better portability between different hardware than SMM rootkits 

(Unified ARC4 execution environment) 



Executable modules found in the AMT memory dump 

(names and numbers taken from their headers) 



LOADER 


: 0x000000. 


.0x0122B8, 


code: 


0x000050. 


.0x0013E0, 


entry: 


0x000050 


KERNEL 


: 0x0122D0. 


.0x28979C, 


code: 


0x012320. 


.0x05F068, 


entry : 


0x03lAl0 


PMHWSEQ 


: 0x2897B0. 


.0x28DDF0, 


code: 


0x289800. 


.0x28CAD8, 


entry: 


0x28A170 


QST 


: 0x28DE00. 


.0x2A79E8, 


code: 


0x28DE50. 


.0x29B3F4, 


entry: 


0x29lB48 


OS 


: 0x2A7A00. 


.0x88EE28, 


code: 


0x2A7A50. 


.0x5ADA48, 


entry: 


0x4ECC58 


ADMIN_CM 


: 0x88EE40. 


.0x98CCF8, 


code: 


0x88EE90. 


.0x91A810, 


entry: 


0x8B2994 


AMT_CM 


: 0x98CDl0. 


.0xAA35FC, 


code: 


0x98CD60. 


.0xA2089C, 


entry: 


0x9BB964 


ASF CM 


: 0xAA3610. 


.0xAB4DEC, 


code: 


0xAA3660. 


.0xAAD59C, 


entry: 


0xAABC58 



01012E60 
01012E64 
01012E68 
01012E6C 
01012E70 
01012E74 
01012E78 
01012E7C 
01012E80 
01012E84 
01012E88 
01012E8C 
01012E90 
01012E94 
01012E98 
01012E9C 
01012EA0 
01012EA4 
01012EA8 



mov . f lp_count , r 2 
or r4, rO, rl 
jz.f [blink] 
and.f 0, r4, 3 
shr r4, r2, 3 
bnz loc_1012EFC 
lsr.f lp_count, r4 
sub rl, rl, 4 
sub r3, rO, 4 
lpnz loc_1012EA8 
Id. a r4, [rl+4] 



Id. 


,a 


r5, 


[rl+4] 


Id. 


,a 


r6, 


[rl+4] 


Id. 


,a 


r7, 


[rl+4] 


St. 


,a 


r4, 


[r3+4] 


St. 


,a 


r5, 


[r3+4] 


St. 


,a 


r6, 


[r3+4] 


St. 


,a 


r7, 


[r3+4] 



This function from the KERNEL 
module is called quite often probably 
by a timer interrupt handler. 



bed loc 1012ED8 



PROGRAMMING uC DMA WITH BARE HANDS 
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Programming internal DMA 

hardware in JTAG debugger to 

copy 64 bytes From 0x73000 host 

phys addr to internal memory 



amaaaec 



---— — — —--— — 



w^e 00000688 : 

w =diuKl ,.8x73888,0, 8x2880093 , 64> 
^rrj-red 64 B of d<iLd fmn Most 8x08Q738O0 
-jil Statu? - 1 

I: Fa 55 8b cc 81 ec 84 00 80 00 39 45 b4 89 5d M 

18= 89 4d be 89 5S cB 89 75 cc 89 7d d8 B£ 28 d0 89 

888028= 45 c4 td 45 ft 58 68 82 JJ Jf 00 e8 b£ 88 00 68 

flflflftlEH: fid 45 44 SB tfl lc £8 AH MM rH ft .S^« 

0000 18: 0888088800880008 88 00 08 0f* 
888858- 88 88 00 88 88 88 88 00 88 88 00 8U 

1= 88 88 88 88 88 08 88 88 00 88 00 8J 
08878: HA BB 88 BB 88 BB 88 AH tift ttfl Hfl ft 

fiiinp WxZBH 

: a c 81)55 Fa B884«c81 45898088 bB5d89b4 



-U E..1 

.H..IK.U. .1.. . 
E - -E-E'ri-Il. . 

_ h . I n.ii. Tr rf TT1 



DMA-cd malicious VM Exit 
handler 



E..1. 



* 1 7Z. r ■ >»**: fc'F3 : ^W [ : ft 1 :i'. 



IrJU^WTTi 
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source: Yuriy Bulygin, Intel, Black Hat USA 2008 



But how to program it? Of course this is not documented 

anywhere... 

(And the rootkit can't just use ARC4 JTAG debugger, of course) 



Idea of how to learn how AMT code does DMA to host memory 



We know that AMT emulates "Virtual CDROM" that 
might be used by remote admin to boot system into 

OS installer... 



...we can also debug the AMT code using function hooking and 



counters... 



AnAMT 
function X. 



Our debugging stubs 

(The counter_* variables are also located in 
the AMT memory -- we read them using the 
remapping trick) 




Most of the functions can be spotted by looking for the following 
prologue signature: 



04 3E 0E 10 



st blink, [sp+4] 



So we can boot off AMT CDROM e.g. a Linux OS and try to access 

the AMT virtual CDROM... 



Q: How is the AMT CDROM presented to BIOS/OS? 
A: As a PCI device... 



root@domO> 



[root@q35 ~]# lspci -s 00:03.2 -v 



00:03.2 IDE interface: IntGl Corporation PT IDER Controll 

er (rGV 02) (prog-if 85 [MastGr SgcO PriO] ) 

Subsystem: IntGl Corporation Unknown dGvicG 4f4a 
Flags: bus master, 66MHz, fast devsel, latency 0, 

IRQ 9 

I/O ports at 2480 [sizG=8; 
I/O ports at 24a4 [size=4; 
I/O ports at 2478 [size=8| 
I/O ports at 24a0 [size=4; 
I/O ports at 2440 [size=16] 

CapabilitiGS : [c8] Power ManagGmGnt version 3 
Capabilities: [d0] Message SignallGd Interrupts: 
Mask- 64bit+ Queue=0/0 Enable- 



[root@q35 ~]# 



We have traced BIOS accesses to AMT CDROM during boot; it 
turned out that BIOS did not use DMA transfers, it used PIO data 

transfers :( 



Fortunately, the above PCI device fully conforms to ATAPI 
specifications; as a result, it is properly handled by the Linux 

ata_generic . ko driver 

(if loaded with all_generic_ide flag) 



* root@f9q35: 



f9q35 kGrnGl 
w) -> IRQ 18 
f9q35 kGrnGl 
f9q35 kGrnGl 
f9q35 kGrnGl 
40 irq 18 
f9q35 kGrnGl 
48 irq 18 
f9q35 kGrnGl 
, 1.00, max 
f9q35 kGrnGl 
f9q35 kGrnGl 
f9q35 kGrnGl 
f9q35 kGrnGl 

1.00 PQ: 
f9q35 kGrnGl 
f9q35 kGrnGl 
f9q35 kGrnGl 

1.00 PQ: 
[root@f 9q35 
[root@f 9q35 
[root@f 9q35 
[root@f 9q35 



: ACPI: PCI IntGrrupt 0000 : 00 : 03 . 2 [C] -> GSI 18 (IgvgI, lo 

: scsi6 : ata_gGnGric 

: scsi7 : ata_gGnGric 

: ata7: PATA max UDMA/100 cmd 0x2480 ctl 0x24a4 bmdma 0x24 

: ata8: PATA max UDMA/100 cmd 0x2478 ctl Ox24aO bmdma 0x24 

: ata7.00: ATAPI : IntGl Virtual LS-120 Floppy UHD Floppy 



ata7 
ata7 
ata7 
scsi 

sd 6 
sd 6 
scsi 



01: | ATAPI: IntGl Vir tual CD, | 1.00, 
: contigurGd tor UDMS7" 
01: configurGd for UDMA/100 
6:0:0:0: DirGct-AccGss IntGl 



max UDMA/100 



Virtual Floppy 



0:0:0: [sdb] AttachGd SCSI rGmovablG disk 
0:0:0: AttachGd scsi gGnGric sg2 typG 
6:0:1:0: CD-ROM IntGl Virtual CD 



We can instruct ata_generic . ko whether to use or not DMA 

for the virtual CDROM accesses 



we can do the diffing between two traces and find out which AMT 

code is responsible for DMA :) 



This way we found (at least one) way to do DMA from AMT to the 

host memory 



struct dmadesc_t { 

unsigned int src_lo; 
unsigned int src_hi; 
unsigned int dst_lo; 

unsigned int dst_hij 

unsigned int count; // SR instruction: Store to Auxiliary Register 

unsigned int resl; void sr(unsigned int addr, unsigned int value) 
unsigned int res2; asm("sr rl, [rO]"); 
unsigned int res3; } 
} dmadesc[ NUMBER OF DMA ENGINES] ; 



/* the id of DMA engine 



void dma_amt 2 host (unsigned int idx, /* the id of : 
unsigned int amt_source_addr, 
unsigned int host_dest_addr, 
unsigned int transfer_length) 

{ 

unsigned int srbase = 0x5010 + 4 * idx; 

memset (&dmadesc[idx] , 0, sizeof dmadesc[idx] ) ; 

dmadesc [ idx ] . src_lo = amt_source_addr ; 

dmadesc[idx] .dst_lo = host_dest_addr; 

dmadesc [idx] .count = transfer_length; 

sr( srbase + 1, &dmadesc[idx] ) ; 

sr (srbase + 2 , ) ; 

sr (srbase + 3, ) ; 

sr(srbase + 0, 0x189); 



10.5.2 CMD— Command Register 



B/D/F/Type: 


0/3/2/PCI 


Address Offset:. 


4 5h 


Default Value: 


OOOOh 


Access: 


RO. R/W 


Size: 


16 bits 



Reset: Host System reset or D3->D0 transition of function. 

This register provides basic control over the device's ability to respond to and perform 
Host system related acesses. 




i n te J Hd fidifCiability C n g i"n e S u tti. y±. Ivm ft a g i'h t e r t, 



Bit 


Access 


Default 
Value 


KS17 PWft 


Description 


2 


ww 


Ob 


Care 


Hue M aster £ n able | &H E 1 : This bit controls the PT 
""unction's ability to act as a master for data transfers. This 
bit does not impact the generation of completions for split 
transaction commands. 


1 


RO 


Ob 


Care 


Memory Space Enable (MSE): PT Function does not 

contain target memory space. 





ftyw 


Ob 


Core 


1 i O Space enable ( 1 OSEJ : This bit controls access to the 
PT function's target I/O space. 



source: intel.com 



Unfortunately, upon reboot, the BME bit for IDER device is 

cleared, which prevents DMA transfers... 



However: rootkit can detect that a 
host reboot is in progress (because 

DMA transfers fail to work), and 
force reboot to AMT CDROM, that 

will set BME bit and resume OS 

boot 



Possibly, using other ME PCI device bypasses the BME 

limitation? 

(there is nothing about BME bit in Yuriy Bulygin's talk on 

DeepWatch from BH US 2008) 

This would allow for SRTM bypass (AMT could inject/replace 

already-measured code while it's executing) 

But we haven't found any other way to do DMA without BME 

so far... 



Hooked AMT 

function that is 

executed periodically 

(regardless of 

whether AMT is 

enabled or not in the 

BIOS) 




Chipset ME/AMT: 

All code executed by 
the chipset's ARC4 
processor, even if the 
host in sleep mode! 



Hypervisor (optional) 



SMM 



DMA access 



Host OS (e.g. Windows) 



Host Memory: 

all code executed 
on the host CPU(s) 



What about VT-d? Can the OS protect itself against AMT rootkit? 



DMA REMAPPING 

* VT-d capable chipsets have one or more DMA-remapping 

engines virtualizing Directed I/O access [12] 

* Internal devices arc also a subject to DMA-remapping 

* Chipset has dedicated register-set for each DMA-remap unit 

accessible by software as MM 10 range which software can use 
to protect certain memory regions from certain I/O devices 

* Rootkit can create DMA-remapping page tables to translate 
addresses of DMA requests issued by embedded uC (identified 
by its PCI B/D/F) to different host, physical addresses 

- or read/write protect entries in DMAr pages tables 

- or mark context- entry as not Present to cause translation fault 

- or enable PLMR/PHMR DMA- protected regions to prevent any DMA 

* And relocate code/data (VMExit handler, VMCS ..) to memory 
protected by DMAr page tables or to PMR regions 



t/zittn 



source: Yuriy Bulygin, Intel, Black Hat USA 2008 



SO WHAT CAN WE DO ABOUT THIS ?? 

• DMA-remapping unit can distinguish DMA requests 
issued by DeepWatch internal device function inside 
embedded uC 

• by its requester id from DMA requests issued by 
other internal functions 

• and not translate them 

• Or disable and lock DMA-remapping of DeepWatch 

device function if DeepWatch is used 

• And allow only trusted software like SMX 
authenticated code modules (Intel® TXTJ to enable 
and program DMA-remap engine for DeepWatch 



VlKN 



source: Yuriy Bulygin, Intel, Black Hat USA 2008 



So, if Intel allowed its AMT/ME code to bypass VT-d (in order to allow 

rootkit detectors in the chipset), then our AMT rootkit would 

automatically gain ability to bypass VT-d as well! 



We have verified that Xen 3.3+ uses VT-d in order to protect its own 
hypervisor and consequently our AMT rootkit is not able to access 

this memory of Xen hypervisor 

(But still, if ME PCI devices are not delegated to a driver domain, then we can access domO memory) 



Still, an AMT rootkit can, if detected that it has an 
opponent that uses VT-d for protection, do the following: 
® Force OS reboot 
^ Force booting from Virtual CDROM 
^ Use its own image for the CDROM that would infect 
the OS kernel (e.g. xen.gz) and disable the VT-d there 



Via Trusted Boot, e.g. SRTM or DRTM (Intel TXT) 

(Keep in mind that we can bypass TXT though, if used without STM, and there is still no 

STM available as of now) 



Final Thoughts 




We do like many of the new Intel technologies (VT-x,VT-d, TXT), ... 



But AMT is different in that it can potentially be greatly 

abused by the attacker 

(VT-d or TXT can potentially be bypassed, but they cannot help the attacker!) 



But keep in mind that our attack doesn't work on the latest Q45 
chipsets - a sign that Intel treats the security seriously... 



You do not want this privileged code to fall into enemy's hand, do you? 




http : / /invisiblethingslab . com 
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